Information storage and processing, electronic recording of sound and images, electronic communications, and electronic entertainment systems have become widespread, and portable applications of these technologies are growing rapidly. The integration of a camera with a head-mounted display was disclosed by Hawthorne in U.S. Pat. No. 4,672,436 for use in camcorder systems. In this case, only the display is placed on the head and the camera is held in the operator""s hand. U.S. Pat. No. 5,491,510 discloses a system for superimposing an image of an object. The camera described in this patent is separate from the user, requiring significant processing to correct for the large parallax angle between the camera and user""s view.
In U.S. Pat. No. 5,129,716, Holakovsky discloses a head mounted stereoscopic vision system for the purpose of providing television audio and video information to the user. U.S. Pat. No. 5,717,479 discloses safety glasses comprising audio speakers, a microphone, and a heads-up display. The speakers are placed in protective ear inserts in order to attenuate external noise. U.S. Pat. No. 5,606,743 to Vogt discloses a radio receiver and speakers mounted in an eyeglass frame, but includes no provision for audio input or for video input or output. U.S. Pat. No. 5,367,345 to daSilva discloses an audio system enclosed in an eyeglass retainer that is substantially separate from any eyeglass system to which it may be attached. Hearing aid audio components have been mounted in eyewear.
Ritchey, in U.S. Pat. Nos. 5,130,794 and 5,495,576, discloses a panoramic display and its use in an audiovisual virtual reality/telepresence system. This system is designed specifically to allow immersive interaction with a remote or virtual object. An electronic camera in the nose frame of eyewear has been offered for sale by one organization that provides covert surveillance devices.
Full portability for many of these interface systems requires reduction to the portable scale of the user interface. For most applications, it is desirable to have an audio/video interface which is compact and allows hands-free or nearly hands-free operation. In addition, it is desirable that such an interface be unobtrusive and not detract from the appearance of the user. The eyeglass interface system of present invention addresses the need for such an interface, by integrating the complete audio/video interface within eyewear.
The eyeglass interface system uses an eyeglass optical system or display assembly such as that in accordance with U.S. patent application Ser. No. 08/844,098, filed on Apr. 18, 1997, now U.S. Pat. No. 5,886,822, the disclosure of which is incorporated by reference herein. More particularly, the eyeglass interface system of the present invention comprises an eyeglass frame having first and second lens holders and first and second temples configured to be supported on a user""s head. The display assembly used in the eyeglass interface system comprises an image source supported by the first temple, an optical pathway disposed internally within the first lens to receive light from the image source, and an insert in the first lens to redirect light on the optical pathway to the user""s eye, as described more particularly in U.S. patent application Ser. No. 08/844,098. An audio and/or video assembly is mounted to the second temple, preferably in communication with the display assembly. The audio and/or video assembly is operative to receive or transmit audio and/or video signals. Interface circuitry is provided in communication with the display assembly and the audio and/or video assembly.
In a typical configuration according to the present invention, the display assembly is mounted within one half of the eyeglasses. The audio and/or video assembly comprises a camera assembly mounted within the other half. The system may also include audio transducers (microphone and/or speakers) which provide audio data and/or which collect sound for recording, digital signal processing and analysis. In this way, the microphone may also provide voice activated commands to the system. Additionally, the eyeglass system may include batteries and an RF or infrared communication system to eliminate the need for a cable to the glasses. Various types of systems may be formed using one or more of the subsystems. In its most complete form, the present eyeglass interface system includes the following subsystems or assemblies: an audio input assembly for collecting speech input from the user, a display assembly, an audio output assembly, a camera assembly, and a head-tracking assembly to provide a computer with the position of the user""s head. Thus, the eyeglass interface system includes audio, video, tracking and power modules.
The eyeglass interface system of the present invention provides computer input and output in highly compact forms, preferably entirely within eyeglasses, which look to the casual observer like ordinary eyewear. In some embodiments of the invention, the subsystems or assemblies are entirely concealed within the eyeglass frame and lenses. The invention comprises not only the integration of the hardware subsystems within an eyeglass frame, but also the integration of the subsystems with hardware and software intended for specific functions.
The present invention is an improvement over devices in the prior art owing to the complete integration of the camera assembly within the eyeglass frame and lens system of the eyeglasses. The mechanical aspects of the camera, including focusing mechanisms, are integrated within the eyeglass temple, as are the electronics in the form of a multi-chip module integrated in the temple. Additionally, lenses and optical surfaces of the camera assembly are integrated in the temple and even in the eyeglass lens itself. In this way, the camera assembly may attain its most compact form, may be made light in weight, and may attain a greater degree of concealment and functionality than prior art devices. In addition, the present invention places the field of view of the camera in proximity to or convergent with the operator""s field of view, so that the camera records the visual field observed by the user. The present invention allows the camera to have the same line of sight as the user""s eye, without parallax error.
The present invention also includes the use of the camera assembly to control a computer. While in prior art camera systems, the camera merely collects data for the computer, in the present invention, the camera assembly also collects input commands. These commands may be in the form of written symbols, hand signals, bar codes, or other forms that cause the computer to execute instructions.
The integration of the camera and display within eyewear according to the present invention provides eyewear that is more compact and therefore easier to wear than prior art devices, and is more aesthetically appealing, because it looks like eyeglasses. The present invention uses optics embedded in the eyeglass lens rather than suspending them in a mount, making the system more durable. The present invention incorporates audio and video inputs as well as outputs. The audio assembly may be integrated into the eyeglass frame. Details of the visual display are not used in conjunction with normal hearing, and therefore one embodiment of the present system integrating an audio assembly seeks to avoid attenuating external sounds. The current system allows the user to interact with computer generated or other electronically relayed information without requiring the occlusion of the actual surroundings.